Microprocessor with Improved Data Stream Prefetching

ABSTRACT

A microprocessor coupled to a system memory by a bus includes an instruction decode unit that decodes an instruction that specifies a data stream in the system memory and a stream prefetch priority. The microprocessor also includes a load/store unit that generates load/store requests to transfer data between the system memory and the microprocessor. The microprocessor also includes a stream prefetch unit that generates a plurality of prefetch requests to prefetch the data stream from the system memory into the microprocessor. The prefetch requests specify the stream prefetch priority. The microprocessor also includes a bus interface unit (BIU) that generates transaction requests on the bus to transfer data between the system memory and the microprocessor in response to the load/store requests and the prefetch requests. The BIU prioritizes the bus transaction requests for the prefetch requests relative to the bus transaction requests for the load/store requests based on the stream prefetch priority.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.11/463,939, filed Aug. 11, 2006, which is a divisional of Ser. No.10/449,825, filed May 30, 2003 (now U.S. Pat. No. 7,177,985 issued Feb.13, 2007), each of which is hereby incorporated by reference in itsentirety.

This application is related to the following U.S. Non-Provisionalapplications:

Ser. No. (Atty. Docket No.) Filing Date Title 10/449,818 May 20, 2003MICROPROCESSOR WITH (MIPS.0186-00- IMPROVED DATA STREAM US) PREFETCHING11/463,957 Aug. 11, 2006 MICROPROCESSOR WITH (MIPS.0186-01- IMPROVEDDATA STREAM US) PREFETCHING 11/463,954 Aug. 11, 2006 MICROPROCESSOR WITH(MIPS.0156-04- IMPROVED DATA STREAM US) PREFETCHING 11/463,950 Aug. 11,2006 MICROPROCESSOR WITH (MIPS.0156-05- IMPROVED DATA STREAM US)PREFETCHING

FIELD OF THE INVENTION

This invention relates in general to the field of prefetching data intoa microprocessor, and more specifically to efficient use of caches whenprefetching data streams.

BACKGROUND OF THE INVENTION

A microprocessor is a digital device that executes instructionsspecified by a computer program. A typical computer system includes amicroprocessor coupled to a system memory that stores programinstructions and data to be processed by the program instructions. Theperformance of such a system is hindered by the fact that the timerequired to fetch data from the system memory into the microprocessor,referred to as memory fetch latency, is typically much larger than thetime required for the microprocessor to execute the instructions thatprocess the data. The time difference is often between one and twoorders of magnitude. Thus, the microprocessor may be sitting idle withnothing to do while waiting for the needed data to be fetched frommemory.

However, microprocessor designers recognized long ago that programs tendto access a relatively small proportion of the data a relatively largeproportion of the time, such as frequently accessed program variables.Programs with this characteristic are said to display good temporallocality, and the propensity for this characteristic is referred to asthe locality of reference principle. To take advantage of thisprinciple, modern microprocessors typically include one or more cachememories. A cache memory, or cache, is a relatively small memoryelectrically close to the microprocessor core that temporarily stores asubset of data that normally resides in the larger, more distantmemories of the computer system, such as the system memory. A cachememory may be internal or external, i.e., may be on the samesemiconductor substrate as the microprocessor core or may be on aseparate semiconductor substrate. When the microprocessor executes amemory access instruction, the microprocessor first checks to see if thedata is present in the cache. If not, the microprocessor fetches thedata into the cache in addition to loading it into the specifiedregister of the microprocessor. Now since the data is in the cache, thenext time an instruction is encountered that accesses the data, the datacan be fetched from the cache into the register, rather than from systemmemory, and the instruction can be executed essentially immediatelysince the data is already present in the cache, thereby avoiding thememory fetch latency.

However, some software programs executing on a microprocessor manipulatelarge chunks of data in a relatively regular and linear fashion, whichmay be referred to as processing of data streams. Examples of suchprograms are multimedia-related audio or video programs that process adata stream, such as audio or video data. Typically, the data stream ispresent in an external memory, such as in system memory or a video framebuffer. Generally speaking, these programs do not demonstrate goodtemporal locality, since the data streams tend to be large, and theindividual data elements in the stream are accessed very few times. Forexample, some programs read in the data stream only once, manipulate it,and write the results back out to another location, without everreferencing the original data stream again. Consequently, the benefitsof the cache are lost, since the memory fetch latency must still beincurred on the first read of the data stream.

To address this problem, several modern microprocessors exploit the factthat that many times the programmer knows he will need the data wellbefore execution of the instructions that actually process the data,such as arithmetic instructions. Consequently, modern microprocessorshave added to or included in their instruction sets prefetchinstructions to prefetch data into a cache of the processor before thedata is needed. Some processors have even included prefetch instructionsthat enable the programmer to define a data stream to be prefetched.Other microprocessors have added hardware to detect a pattern of a datastream being accessed and begin prefetching into the data cacheautomatically. Prefetching enables the microprocessor to perform otheruseful work while the data is being prefetched from external memory inhopes that the data will be in the cache by the time the instructionthat needs the data is executed.

However, current prefetching techniques still suffer drawbacks, and theneed for improved prefetching performance is constantly increasing dueto the proliferation of multimedia data streams and because memorylatency is becoming longer relative to microprocessor execution speed.

SUMMARY

The present invention provides a microprocessor and method for improvingdata stream prefetching through a hybrid hardware/software approach.

In one aspect, the present invention provides a microprocessor coupledto a system memory by a bus. The microprocessor includes an instructiondecode unit, for decoding an instruction. The instruction specifies adata stream in the system memory and a stream prefetch priority. Themicroprocessor also includes a load/store unit, coupled to theinstruction decode unit, for generating load/store requests to transferdata between the system memory and the microprocessor. Themicroprocessor also includes a stream prefetch unit, coupled to theinstruction decode unit, for generating a plurality of prefetch requeststo prefetch the data stream from the system memory into themicroprocessor. The prefetch requests specify the stream prefetchpriority. The microprocessor also includes a bus interface unit (BIU),coupled to the stream prefetch unit and the load/store unit, forgenerating transaction requests on the bus to transfer data between thesystem memory and the microprocessor in response to the load/storerequests and the prefetch requests. The BIU prioritizes the bustransaction requests for the prefetch requests relative to the bustransaction requests for the load/store requests based on the streamprefetch priority.

In another aspect, the present invention provides a method forprefetching a data stream into a microprocessor from a system memorycoupled to the microprocessor by a bus. The method includes decoding aninstruction. The instruction specifies the data stream and a streamprefetch priority. The method also includes generating a plurality ofload requests to load data from the system memory into themicroprocessor in response to execution of a plurality of loadinstructions. The method also includes generating a plurality ofprefetch requests to load portions of the data stream from the systemmemory into the microprocessor in response to the decoding. The methodalso includes prioritizing the prefetch requests relative to the loadrequests for transmission on the bus based on the stream prefetchpriority.

In another aspect, the present invention provides a computer programproduct for use with a computing device, the computer program productincluding a computer usable storage medium, having computer readableprogram code embodied in the medium, for causing a microprocessorcoupled to a system memory by a bus. The computer readable program codeincludes first program code for providing an instruction decode unit,for decoding an instruction. The instruction specifies a data stream inthe system memory and a stream prefetch priority. The computer readableprogram code also includes second program code for providing aload/store unit, coupled to the instruction decode unit, for generatingload/store requests to transfer data between the system memory and themicroprocessor. The computer readable program code also includes thirdprogram code for providing a stream prefetch unit, coupled to theinstruction decode unit, for generating a plurality of prefetch requeststo prefetch the data stream from the system memory into themicroprocessor. The prefetch requests specify the stream prefetchpriority. The computer readable program code also includes fourthprogram code for providing a bus interface unit (BIU), coupled to thestream prefetch unit and the load/store unit, for generating transactionrequests on the bus to transfer data between the system memory and themicroprocessor in response to the load/store requests and the prefetchrequests. The BIU prioritizes the bus transaction requests for theprefetch requests relative to the bus transaction requests for theload/store requests based on the stream prefetch priority.

Other features and advantages of the present invention will becomeapparent upon study of the remaining portions of the specification anddrawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a microprocessor according to the presentinvention.

FIG. 2 is a block diagram of the stream prefetch unit of FIG. 1according to the present invention.

FIG. 3 is a block diagram of a stream prefetch engine of FIG. 2according to the present invention.

FIG. 4 is a block diagram illustrating a stream prefetch instruction ofFIG. 1 according to the present invention.

FIG. 5 is a block diagram illustrating four embodiments of the operandfield of the stream prefetch instruction of FIG. 4 according to thepresent invention.

FIG. 6 is a block diagram illustrating the format of a stream descriptoraccording to one embodiment of the present invention.

FIG. 7 is a block diagram illustrating the format of a halt streaminstruction according to the present invention.

FIG. 8 is a block diagram illustrating thestream_prefetch_priority_parameters of FIG. 6 according to the presentinvention.

FIG. 9 is a block diagram illustrating an example data stream templatespecified by a stream descriptor of FIG. 6 according to the presentinvention.

FIG. 10 is a block diagram illustrating conditions which selectivelytrigger prefetching of a data stream with the stream template example ofFIG. 9 according to the present invention.

FIG. 11 is a flowchart illustrating stream prefetching according to thepresent invention.

FIG. 12 is a flowchart illustrating in detail block 1126 of FIG. 11according to the present invention.

FIG. 13 is a flowchart illustrating in detail block 1126 of FIG. 11according to an alternate embodiment of the present invention.

FIG. 14 is a flowchart illustrating operation of the microprocessor inresponse to a TLB miss in the memory subsystem of a stream prefetchrequest of FIG. 1 according to the present invention.

FIG. 15 is a flowchart illustrating operation of the microprocessor inresponse to a page fault caused by a stream prefetch request of FIG. 1according to the present invention.

FIG. 16 is a flowchart illustrating operation of the microprocessor inresponse to a protection fault caused by a stream prefetch request ofFIG. 1 according to the present invention.

FIG. 17 is a block diagram of portions of the memory subsystem of FIG. 1having a separate stream prefetch TLB according to the presentinvention.

FIG. 18 is a block diagram of portions of the memory subsystem of FIG. 1having a separate stream prefetch TLB according to an alternateembodiment of the present invention.

FIG. 19 is a block diagram of portions of the memory subsystem of FIG. 1having a separate stream prefetch TLB according to an alternateembodiment of the present invention.

FIG. 20 is a flowchart illustrating operation of the stream hit detectorof FIG. 3 according to the present invention.

FIG. 21 is a block diagram of the stream hit detector of FIG. 3according to one embodiment of the present invention.

FIG. 22 is a flowchart illustrating in detail block 1202 of FIG. 12according to the present invention.

FIG. 23 is a flowchart illustrating in detail block 1202 of FIG. 12according to the present invention is shown.

DETAILED DESCRIPTION

Referring now to FIG. 1, a block diagram of a microprocessor 100according to the present invention is shown.

Microprocessor 100 includes a processor bus 132. Processor bus 132couples microprocessor 100 to other devices in a computer system,including a system memory, such as dynamic random access memory (DRAM).In particular, the system memory stores data streams, whichmicroprocessor 100 prefetches according to the present invention.Typically, the system memory stores program instructions and dataexecuted by microprocessor 100. However, the system memory should beunderstood as encompassing any memory that stores a data stream that canbe prefetched by microprocessor 100. A data stream is a set of bytesdefined by a base memory address and an end memory address. The endmemory address may be defined by a length added to the base address. Thedata stream may or may not include all the data bytes between the baseand end addresses. Rather, the data stream may comprise a subset of thebytes between the base and end addresses. The base address may be aphysical address or a virtual address. In an alternate embodiment, adata stream is a set of stream blocks defined by one or more streamblock sizes, and one or more stream block stride distances between thestream blocks, and a base memory address specifying the first streamblock. The individual data structure elements of a data stream that areprocessed may comprise various word sizes, including a single byte, atwo-byte word, four-byte word, or any other number of bytes. Processorbus 132 comprises a medium for transmitting addresses, data, and controlinformation between microprocessor 100 and the system memory. In oneembodiment, processor bus 132 comprises a bus conforming substantiallyto the MIPS R10000 microprocessor bus. In one embodiment, processor bus132 comprises a bus conforming substantially to the HyperTransport™ I/OLink Specification Revision 1.05.

Microprocessor 100 also includes a bus interface unit 102 coupled toprocessor bus 132. Bus interface unit 102 performs bus transactions onprocessor bus 132 to transfer data between microprocessor 100 and othersystem devices, such as the system memory. In particular, bus interfaceunit 102 performs bus transactions on processor bus 132 to prefetchportions of a data stream from the system memory into microprocessor 100in response to a stream prefetch instruction specifying the data streamaccording to the present invention. In one embodiment, bus interfaceunit 102 is configured to combine multiple requests to performtransactions on processor bus 132 into one or more larger bustransaction requests on processor bus 132.

Microprocessor 100 also includes a memory subsystem 104 coupled to businterface unit 102. Memory subsystem 104 comprises one or more cachememories and associated control logic for caching instructions and datafrom the system memory. In the embodiment shown in FIG. 1, memorysubsystem 104 comprises a level-1 (L1) instruction cache 156, an L1 datacache 158, a unified level-2 (L2) cache 154 backing up the two L1 caches156 and 158, and a level-3 cache 152, backing up L2 cache 154. Inanother embodiment, memory subsystem 104 also includes a dedicatedprefetch buffer, for buffering prefetched cache lines from the systemmemory. A cache line is the smallest unit of data that can betransferred between the system memory and a cache of memory subsystem104. In one embodiment, a cache line comprises 32 bytes. In oneembodiment, L1 instruction cache 156 and L1 data cache 158 each comprisean 8 KB cache, L2 cache 154 comprises a 64 KB cache, and L3 cache 152comprises a 512 KB cache. In one embodiment, L3 cache 152 comprises thehighest level and L1 data cache 158 comprises the lowest level of thecache hierarchy of memory subsystem 104.

In the embodiment of FIG. 1, bus interface unit 102 includes a requestqueue 144, or request buffer 144, for storing requests made by memorysubsystem 104 to perform a transaction on processor bus 132. Eachrequest in request queue 144 includes information specifying thecharacteristics of the request. Bus interface unit 102 also includes anarbiter 142, coupled to request queue 144, which prioritizes therequests based on the request characteristics and issues transactions onprocessor bus 132 based on the prioritization. In one embodiment,translation look-aside buffers (TLBs) in memory subsystem 104, such asTLBs 1702, 1704, 1802, and 1902A-D of FIGS. 17 through 19, holdattribute bits associated with each memory page in the address space ofmicroprocessor 100. In one embodiment, the attribute bits specify thebus transaction priority for loads, stores, and stream prefetches fromthe memory page. In one embodiment, the stream prefetch priorityattributes held in the TLBs are populated based in part on astream_prefetch_priority indicator 814, described below with respect toFIG. 8. In one embodiment, when memory subsystem 104 makes a request tobus interface unit 102, memory subsystem 104 provides the TLB attributebits to bus interface unit 102. In one embodiment, when memory subsystem104 makes a request to bus interface unit 102, memory subsystem 104 usesthe TLB attribute bits to generate the priority of the request to businterface unit 102. In one embodiment, when memory subsystem 104 makes arequest to bus interface unit 102, memory subsystem 104 generates thepriority of the request to bus interface unit 102 based on a taskpriority, described below with respect to FIG. 8.

Microprocessor 100 also includes an instruction fetch unit 112 coupledto memory subsystem 104. Instruction fetch unit 112 fetches programinstructions from L1 instruction cache 156. If the requested instructionis missing in L1 instruction cache 156, L2 cache 154, and L3 cache 152,then memory subsystem 104 requests bus interface unit 102 to fetch acache line including the missing instruction from the system memory. Inparticular, instruction fetch unit 112 fetches load instructions, storeinstructions, stream prefetch instructions, and stream halt instructionsaccording to the present invention.

Microprocessor 100 also includes an instruction decode/dispatch unit 122coupled to instruction fetch unit 112. Instruction decode/dispatch unit122 decodes program instructions provided by instruction fetch unit 112and dispatches the decoded instructions to the appropriate functionalunits of microprocessor 100 for execution. In particular, instructiondecode/dispatch unit 122 decodes and dispatches load instructions, storeinstructions, stream prefetch instructions, and halt streaminstructions, according to the present invention.

Microprocessor 100 also includes a branch unit 114. Branch unit 114receives branch instructions from instruction decode/dispatch unit 122and communicates with instruction fetch unit 112 to control program flowby altering the address at which instruction fetch unit 112 fetchesinstructions. In one embodiment, branch unit 114 includes branchprediction logic for predicting the outcome and target address of abranch instruction.

Microprocessor 100 also includes a register file 124 coupled toinstruction decode/dispatch unit 122. Register file 124 includes aplurality of general purpose registers for use by programs executing onmicroprocessor 100. Register file 124 also includes special purposeregisters for controlling the state of microprocessor 100. Inparticular, register file 124 holds instruction operands and results ofstream prefetch instructions according to the present invention.

Microprocessor 100 also includes a plurality of execution units 126coupled to instruction decode/dispatch unit 122. In one embodiment,execution units 126 include an integer unit and a floating point unit,for performing integer and floating point arithmetic or logicaloperations, respectively. In particular, execution units 126 performarithmetic and logical operations on data elements in data streamsprefetched according to the present invention.

Microprocessor 100 also includes a load/store unit 116 coupled toinstruction decode/dispatch unit 122 and memory subsystem 104. A loadinstruction loads data specified by a memory address into a register ofregister file 124. A store instruction stores data from a register ofregister file 124 to a specified memory address. Load/store unit 116receives load and store instructions from instruction decode/dispatchunit 122 and issues one or more load/store requests 134 to memorysubsystem 104 to load data from memory subsystem 104 (or from the systemmemory if the specified data is not present in memory subsystem 104)into register file 124 or to store data from register file 124 to memorysubsystem 104 (or the system memory). In particular, a load instructionmay be executed to transfer a portion of a data stream from the systemmemory into register file 124 for processing by execution units 126.Advantageously, the cache line of the data stream specified by the loadinstruction may be present in memory subsystem 104 when the loadinstruction executes due to execution of a stream prefetch instructionaccording to the present invention as described below, thereby obviatingthe need to fetch the data from the system memory and avoiding itsaccompanying latency, as would be required if the data were missing inmemory subsystem 104. Similarly, a store instruction may be executed totransfer data processed by execution units 126 from register file 124 toa portion of a data stream in the system memory. Advantageously, thecache line of the data stream specified by the store instruction may bepresent in memory subsystem 104 when the store instruction executes dueto execution of a stream prefetch instruction according to the presentinvention as described below, thereby obviating, in a write-back cacheconfiguration, the need to immediately write the data to the systemmemory and avoiding its accompanying latency, as would be required ifthe data were missing in memory subsystem 104. Load/store request 134includes an indication of whether the request 134 is a load or store, amemory address of the specified data, and the amount of data to beloaded into or stored from microprocessor 100. In one embodiment,load/store request 134 also includes a priority value for use inprioritizing the load/store request 134 relative to stream prefetchrequests 136 and to other load/store requests 134.

Microprocessor 100 also includes a stream prefetch unit 118 coupled toinstruction decode/dispatch unit 122 and memory subsystem 104. Streamprefetch unit 118 receives a stream prefetch instruction 138 accordingto the present invention from instruction decode/dispatch unit 122 andissues a stream prefetch request 136 to memory subsystem 104 in responsethereto. Stream prefetch instruction 138 includes a stream prefetchinstruction opcode 402 and a stream descriptor 600, as described indetail with respect to FIGS. 4, 6, and 8 below. As described below withrespect to FIG. 2, stream prefetch request signal 136 comprises aplurality of stream prefetch request 136 signals from a correspondingplurality of stream prefetch engines 202 of stream prefetch unit 118.Stream prefetch request 136 includes a memory address, namelycurrent_prefetch_addr 324 described below with respect to FIG. 3, forspecifying a location in the system memory from which to prefetch aportion of a data stream specified by stream prefetch instruction 138.In one embodiment, stream prefetch request 136 prefetches one or morecache lines containing the specified memory address. Stream prefetchrequest 136 also includes values of various portions of stream prefetchinstruction 138, such as stream_priority_parameters 614 described belowwith respect to FIG. 6. Stream prefetch unit 118 also receivesload/store request 134 from load/store unit 116 in order to monitor loadand store requests to determine whether a load/store request 134 hitswithin a data stream specified by stream prefetch instruction 138,thereby advantageously enabling stream prefetch unit 118 to prefetch thedata stream in a manner synchronized with program execution of loads (orstores, or both) accessing the data stream, as described below. In oneembodiment, stream prefetch unit 118 also receives a halt streaminstruction 700, as described below with respect to FIG. 7, frominstruction decode/dispatch unit 122 for halting stream prefetch unit118 from prefetching a data stream specified by a previously executedstream prefetch instruction 138. Stream prefetch unit 118 and streamprefetch instruction 138 are described in detail below with respect tothe remaining Figures.

Referring now to FIG. 2, a block diagram of stream prefetch unit 118 ofFIG. 1 according to the present invention is shown.

Stream prefetch unit 118 includes a stream engine allocator 204 and aplurality of stream prefetch engines 202 coupled to stream engineallocator 204. The embodiment of FIG. 2 shows four stream prefetchengines 202, denoted 202A, 202B, 202C, and 202D, which generate streamprefetch requests 136A, 136B, 136C, and 136D, respectively, referred toas stream prefetch request 136 in FIG. 1. Stream engine allocator 204maintains a status of each stream prefetch engine 202 regarding whetherthe stream prefetch engine 202 is currently in use, i.e., whether thestream prefetch engine 202 has been allocated by a currently executingstream prefetch instruction 138, or whether the stream prefetch engine202 is free for allocation. Stream engine allocator 204 receives streamprefetch instruction 138 of FIG. 1. In response, stream engine allocator204 determines whether a stream prefetch engine 202 is free, and if so,allocates a free stream prefetch engine 202 for the stream prefetchinstruction 138 and returns an identifier to a stream prefetch engine202A, 202B, 202C, or 202D in a predetermined register of register file124 of FIG. 1 as the result of the stream prefetch instruction 138. Thatis, stream engine allocator 204 updates the status of the allocatedstream prefetch engine 202 to indicate that the allocated streamprefetch engine 202 is now in use. Stream engine allocator 204subsequently forwards the stream prefetch instruction 138 to theallocated stream prefetch engine 202. Hence, in the embodiment of FIG.2, microprocessor 100 can simultaneously execute up to four distinctstream prefetch instructions 138.

Each stream prefetch engine 202 receives load/store request 134 of FIG.1 for monitoring whether a load/store request 134 hits in the datastream specified by the stream prefetch instruction 138 to which thestream prefetch engine 202 is allocated. In response to the streamprefetch instruction 138 forwarded from stream engine allocator 204 andin response to load/store request 134 hitting in the data streamspecified by the stream prefetch instruction 138, a stream prefetchengine 202 generates a stream prefetch request 136 to memory subsystem104 of FIG. 1 to prefetch portions of the specified data stream asdescribed below. The stream prefetch engines 202 are described in detailbelow with respect to the remaining Figures.

Referring now to FIG. 3, a block diagram of a stream prefetch engine 202of FIG. 2 according to the present invention is shown.

Stream prefetch engine 202 includes control logic 334 that receivesstream prefetch instruction 138 and load/store request 134 of FIG. 1.Control logic 334 comprises combinatorial and sequential logic thatgenerates stream prefetch requests 136 of FIG. 1 in response to streamprefetch instruction 138, load/store requests 134, and other inputsdescribed below.

Stream prefetch engine 202 also includes six registers, 302, 304, 306,308, 312, and 314, referred to collectively as stream descriptorregisters 362, for storing six corresponding fields 602, 604, 606, 608,612, and 614, respectively, of a stream descriptor 600, which isdescribed below with respect to FIG. 6, specified by stream prefetchinstruction 138. Each of the stream descriptor registers 362 providesits contents to control logic 334. The remainder of FIG. 3 will bedescribed after a description of FIG. 6.

Referring now to FIG. 6, a block diagram illustrating the format of astream descriptor 600 according to one embodiment of the presentinvention is shown.

Stream descriptor 600 includes a stream_base field 602 that specifiesthe base memory address, i.e., the starting address, of the data stream.In one embodiment, the stream_base 602 is a virtual address. In oneembodiment, the stream_base 602 is a physical address.

Stream descriptor 600 also includes a stream_length field 604 thatspecifies the difference between the end address of the stream, i.e.,the memory address of the last byte of the data stream, and thestream_base 602. That is, the stream_length 604 specifies the number ofbytes in memory between the first byte of the data stream and the lastbyte of the data stream. However, the stream_length 604 does notnecessarily equal the number of bytes in the data stream, since a datastream may be specified as a subset of the bytes between the stream_base602 and the stream end address. In one embodiment, if the programmerspecifies a stream_length 604 value of 0, then the data stream isunbounded, and the stream prefetch engine 202 synchronously prefetchesthe data stream, by monitoring loads and stores as described herein,until halted by execution of a halt stream instruction, described belowwith respect to FIG. 7.

Stream descriptor 600 also includes a stream_block_size field 608 thatspecifies the size of a stream block. In one embodiment, thestream_block_size field 608 specifies the number of bytes included in astream block. A stream block comprises a contiguous set of bytes withinthe data stream. If a load/store request 134 specifies a location withina stream block of a data stream specified by stream descriptor 600, thenthe load/store request 134 hits in the data stream.

Stream descriptor 600 also includes a stream_block_stride field 606 thatspecifies the periodic distance between stream blocks. That is, thestream_block_stride 606 specifies the number of bytes between the firstbyte of a stream block and the first byte of the next adjacent streamblock. Thus, stream descriptor 600 advantageously enables the programmerto specify a data stream which is a sparse subset of, or a discontinuoustemplate on, a contiguous set of bytes in memory. FIG. 9 illustrates anexample data stream template specified by stream descriptor 600.

Stream descriptor 600 also includes a stream_fetch-ahead_distance 612.The stream prefetch engines 202 monitor load/store requests 134 that hitin their respective data streams and attempt to stay at least the numberof bytes specified by the stream_fetch-ahead_distance 612 ahead of thecurrent_stream_hit_addr 322 of FIG. 3, as described below with respectto FIGS. 10 through 13, thereby synchronizing prefetching of the datastream with program execution. That is, stream prefetch engine 202suspends data stream prefetching when the current_stream_hit_addr 322 isat least the stream_fetch-ahead_distance 612 behind thecurrent_prefetch_addr 324, and resumes data stream prefetching when thecurrent_stream_hit_addr 322 is less than the stream_fetch-ahead_distance612 behind the current_prefetch_addr 324, as described below.

Stream descriptor 600 also includes astream_prefetch_priority_parameters field 614. Thestream_prefetch_priority_parameters field 614 specifies a plurality ofparameters used by microprocessor 100 to prioritize use of memorysubsystem 104 and data stream prefetch requests relative to other memoryaccesses within microprocessor 100. Thestream_prefetch_priority_parameters 614 are described in detail belowwith respect to FIG. 8.

Referring again to FIG. 3, stream prefetch engine 202 also includes astream hit detector 332 coupled to control logic 334. Stream hitdetector 332 receives the contents of stream_base register 302,stream_length register 304, stream_block_stride register 306, andstream_block_size register 308. Stream hit detector 332 also receivesload/store request 134. In response to its inputs, stream hit detector332 generates a hit_in_stream signal 342, which is provided to controllogic 334. Stream hit detector 332 generates a true value onhit_in_stream signal 342 if the address of a load/store request 134 hitsin the data stream specified by stream prefetch instruction 138 instream descriptor 600, as discussed below. That is, stream hit detector332 generates a true value on hit_in_stream signal 342 if the address ofa load/store request 134 specifies the address of a byte in systemmemory included in the data stream specified by stream prefetchinstruction 138 stream descriptor 600. Otherwise, stream hit detector332 generates a false value on hit_in_stream signal 342. The operationof stream hit detector 332 is described below with respect to FIG. 20.One embodiment of stream hit detector 332 is described below withrespect to FIG. 21.

Stream prefetch engine 202 also includes a current_stream_hit_addrregister 322, coupled to control logic 334, which holds the address ofthe most recent load/store request 134 that hit in the data streamspecified by the stream prefetch instruction 138.

Stream prefetch engine 202 also includes a current_prefetch_addrregister 324, coupled to control logic 334, which holds the address ofthe next element of the data stream to be prefetched, i.e., the cacheline implicated by the address will be prefetched into memory subsystem104.

Stream prefetch engine 202 also includes a current_stream_block_startregister 326, coupled to control logic 334, which holds the startingaddress of the stream block currently being prefetched, i.e., thatencompasses current_prefetch_addr 324. The size of a stream block isdefined by stream_block_size 608 of FIG. 6 held in stream_block_sizeregister 308.

Stream prefetch engine 202 also includes a subtractor 352 coupled tocurrent_stream_hit_addr register 322 and current_prefetch_addr register324. Subtractor 352 subtracts current_stream_hit_addr 322 fromcurrent_prefetch_addr 324 to generate a current_fetch-ahead_distance344, which is provided to control logic 334.

Referring now to FIG. 4, a block diagram illustrating stream prefetchinstruction 138 of FIG. 1 according to the present invention is shown.

In the embodiment shown in FIG. 4, stream prefetch instruction 138includes an opcode field 402 and an operand field 404. Opcode 402includes a predetermined value within the opcode space of microprocessor100 which instruction decode/dispatch unit 122 of FIG. 1 decodes as astream prefetch instruction. In one embodiment, a first predeterminedvalue of opcode 402 specifies a stream prefetch instruction inanticipation of loads from the data stream, and a second predeterminedvalue of opcode 402 specifies a stream prefetch instruction inanticipation of stores to the data stream. If a stream prefetch for loadinstruction is specified in opcode 402, the cache lines prefetched intomemory subsystem 104 are initialized with a cache coherency state ofshared. If a stream prefetch for store instruction is specified inopcode 402, the cache lines prefetched from the data stream are broughtinto the specified cache of the memory subsystem 104 with a cachecoherency state of exclusive-unmodified. Advantageously, the streamprefetch for store instruction avoids the latency associated with atransaction on processor bus 132 of FIG. 1 to transition the implicatedcache line from shared to exclusive state when a subsequent storeoperation modifies a previously prefetched cache line of the datastream.

Stream prefetch instruction 138 also includes an operand field 404following opcode field 402. The operand field 404 is used to specify thestream descriptor 600 of FIG. 6. The operand field 404 specifies thestream descriptor 600 according to one of the four embodiments shown inFIG. 5. In embodiment 1 of FIG. 5, the operand field 404 holds thestream descriptor itself, denoted 502, as immediate data. In embodiment2 of FIG. 5, a load instruction executed before the stream prefetchinstruction 138 loads the stream descriptor into a general purposeregister of register file 124 of FIG. 1, and the operand field 404 holdsa register identifier, denoted 504, that identifies the register holdingthe stream descriptor. In embodiment 3 of FIG. 5, the stream descriptoris stored in system memory by the program prior to execution of thestream prefetch instruction 138, and the operand field 404 holds apointer or address, denoted 506, to the stream descriptor in the systemmemory. In embodiment 4 of FIG. 5, the stream descriptor is stored insystem memory by the program prior to execution of the stream prefetchinstruction 138, a load instruction executed before the stream prefetchinstruction loads the system memory address of the stream descriptorinto a general purpose register of register file 124, and the operandfield 404 holds a register identifier, denoted 506, that identifies theregister holding the stream descriptor address. In one embodiment, adifferent predetermined opcode 402 value exists within the instructionset opcode space for differentiating between the different embodimentsfor specifying the stream descriptor of FIG. 5.

The stream prefetch instruction 138 returns a stream_ID value thatspecifies which of the plurality of stream prefetch engines 202 wasallocated to the stream prefetch instruction 138. If no stream prefetchengine 202 is free, then a predetermined value is returned by the streamprefetch instruction 138. In one embodiment, the predetermined value is0, and values 1 through N are returned to specify one of the N streamprefetch engines 202 allocated by the stream prefetch instruction 138.In one embodiment, the stream_ID is returned in a predetermined one ofthe general purpose registers of register file 124 of FIG. 1.

In one embodiment, stream engine allocator 204 also stores an identifierspecifying the currently executing task that executed the streamprefetch instruction 138. The task identifier is used by stream engineallocator 204 and the operating system executing on microprocessor 100to save and restore the state of the allocated stream prefetch engine202 between task switches by the operating system.

In one embodiment, stream prefetch instruction 138 is a hint tomicroprocessor 100. That is, stream prefetch instruction 138 does notaffect the architectural state of microprocessor 100. The correctfunctional operation of the program executing stream prefetchinstruction 138 does not depend upon whether or not the data streamspecified by stream prefetch instruction 138 has been successfullyprefetched, although the performance of the program may be affectedthereby. Consequently, microprocessor 100 performs stream prefetchinstruction 138 on a best-effort basis. For example, in one embodiment,if bus interface unit 102 is busy servicing other program instructions,such as loads, stores, or instruction fetches, then stream prefetchrequests 136 are delayed until bus interface unit 102 is no longer busy.Similarly, in one embodiment, loads, stores, and instruction fetches aregiven higher priority within memory subsystem 104 over stream prefetchinstructions 138.

Referring now to FIG. 7, a block diagram illustrating the format of ahalt stream instruction 700 according to the present invention is shown.The halt stream instruction 700 includes an opcode field 702 and aregister identifier field 704. Opcode 702 includes a predetermined valuewithin the opcode space of microprocessor 100 which instructiondecode/dispatch unit 122 of FIG. 1 decodes as a halt stream instruction.The register identifier 704 specifies a register that is previouslyloaded with a stream_ID value that specifies which of the streamprefetch engines 202 of FIG. 2 is to be halted from prefetching itscurrent data stream. The stream_ID returned by the stream prefetchinstruction 138 is used to populate the register specified by theregister identifier field 704 of the stream halt instruction. The haltedstream prefetch engine 202 specified by the stream_ID in the registerspecified by the register identifier 704 is returned to the free pool ofstream prefetch engines 202 for allocation by stream engine allocator204 to a subsequent stream prefetch instruction.

Referring now to FIG. 8, a block diagram illustratingstream_prefetch_priority_parameters 614 of FIG. 6 according to thepresent invention is shown.

Stream_prefetch_priority_parameters 614 include a cache_level indicator802. Cache_level indicator 802 specifies which level of the cachehierarchy of memory subsystem 104 the cache lines of the prefetched datastream are to be brought into. In one embodiment, a value of 1 incache_level indicator 802 specifies L1 data cache 158 of FIG. 1, a valueof 2 in cache_level indicator 802 specifies L2 cache 154 of FIG. 1, avalue of 3 in cache_level indicator 802 specifies L3 cache 152 ofFIG. 1. In one embodiment, a value of 4 in cache_level indicator 802specifies a prefetch buffer (not shown) in memory subsystem 104. In oneembodiment, a value of 5 in cache_level indicator 802 specifies L1instruction cache 156 of FIG. 1. In one embodiment, a value of 0 incache_level indicator 802 specifies that no cache level is specified.Advantageously, cache_level indicator 802 enables the programmer toefficiently use the memory subsystem 104 based on the localitycharacteristics of the data stream. For example, if the data stream willbe accessed many times within a section of the program, the programmermay wish to place the data stream into the L1 data cache 158, whereas ifthe data stream will only be accessed once or twice as the programpasses through the data stream, the programmer may wish to place thedata stream into the L2 cache 154 or L3 cache 152, in order to avoidreplacing other more frequently used data in the L1 data cache 158.Advantageously, a programmer also, if he knows the configuration of thehierarchy of the cache memories in memory subsystem 104 and the size ofeach cache, can tailor the stream_fetch-ahead_distance 612 andcache_level indicator 802 to avoid wasting memory bandwidth andoverrunning the specified cache, thereby avoiding needlessly evictingother useful data, including prefetched stream data.

The value of cache_level indicator 802 is forwarded to memory subsystem104 in stream prefetch request 136. In one embodiment, if streamprefetch request 136 misses in the level of the memory subsystem 104hierarchy specified by cache_level indicator 802 but hits in a differentlevel of the memory subsystem 104 hierarchy, then memory subsystem 104moves the data to the level specified by cache_level indicator 802. Inone embodiment, if stream prefetch request 136 misses in the level ofthe memory subsystem 104 hierarchy specified by cache_level indicator802 but hits in a different level of the memory subsystem 104 hierarchy,then memory subsystem 104 leaves the data in its current level. Ifstream prefetch request 136 misses in the memory subsystem 104 hierarchyaltogether, then memory subsystem 104 generates a request to businterface unit 102 to fetch the missing cache line.

Stream_prefetch_priority_parameters 614 also include a localityindicator 804. Locality indicator 804 is an alternative to thecache_level indicator 802 for specifying the locality characteristics ofthe data stream. The programmer places a value of 0 in the cache_levelindicator 802 when using locality indicator 804 to specify data streamprefetch characteristics. Locality indicator 804 enables the programmerto abstractly provide his intention of how the data stream should beprefetched into the memory subsystem 104, but leaves the decision to themicroprocessor 100 to map the intentions specified by the programmer tothe particular cache hierarchy embodied in the microprocessor 100executing the stream prefetch instruction 138. Locality indicator 804alleviates the need for the programmer to understand the intricacies ofthe memory subsystem 104 for each version of microprocessor 100 andfacilitates compatibility and improved performance across a wide rangeof versions of microprocessor 100. This is particularly advantageousbecause a programmer may write a program that will be executed ondifferent versions of microprocessor 100 that have different memorysubsystem 104 configurations, and further, the program may execute onversion of microprocessor 100 yet to be produced. For example, oneversion of microprocessor 100 may have an L1 data cache 158, an L2 cache154, and an L3 cache 152; whereas another version of microprocessor 100may only have an L1 data cache 158, an L2 cache 154. Similarly, forexample, the L1 data cache 158 of one version of microprocessor 100 maybe 32 KB; whereas the L1 data cache 158 of another version ofmicroprocessor 100 may be only 8 KB.

In one embodiment, locality indicator 804 includes an urgency field forspecifying the urgency of the data stream. In one embodiment, an urgencyfield value of 0 indicates the data is urgent and should be brought intoas low a level of the cache hierarchy as is reasonable; an urgency fieldvalue of 1 indicates the data is moderately urgent and should be broughtinto a middle level of the cache hierarchy if reasonable; and an urgencyfield value of 2 indicates the data is not urgent and should be broughtinto as high a level of the cache hierarchy as is reasonable.

In one embodiment, locality indicator 804 includes an ephemerality fieldfor specifying the ephemerality of the data stream. In one embodiment,an ephemerality field value of 0 indicates the data is very ephemeraland should be brought into memory subsystem 104 and marked for earlyeviction. In one embodiment, bringing the prefetched cache line into thecache for early eviction comprises setting the prefetched cache line asthe least-recently-used way in a set associative cache that employs aleast-recently-used (LRU) replacement policy. An ephemerality fieldvalue of 1 indicates the data stream should be brought into memorysubsystem 104 and treated normally by the cache replacement policy,whatever the replacement policy is. An ephemerality field value of 2indicates the data stream is highly persistent, and memory subsystem 104should be brought into memory subsystem 104 and marked for lateeviction, i.e., memory subsystem 104 should attempt to evict other cachelines before evicting this cache line. In one embodiment, bringing theprefetched cache line into the cache for late eviction comprises settingthe prefetched cache line as the most-recently-used way in a setassociative cache that employs a least-recently-used (LRU) replacementpolicy.

In one embodiment, control logic 334 specifies a cache level andeviction policy in stream prefetch request 136 based on localityindicator 804 or cache_level 802, stream_fetch-ahead_distance 612, andthe configuration of memory subsystem 104 to advantageously avoid memoryfetch latency while keeping the memory subsystem 104 as clean aspossible.

Stream_prefetch_priority_parameters 614 also include three fields forspecifying a policy for responding to three distinct abnormal accessesto a TLB in memory subsystem 104 of FIG. 1. In a normal TLB access, theTLB is accessed with a virtual page address, and the TLB looks up thepage address and finds the page address cached therein, i.e., the pageaddress hits in the TLB. The first abnormal TLB access is a TLB miss,i.e., the virtual page address is not cached in the TLB. In a normal TLBaccess, the TLB provides cached TLB information associated with thememory page specified by the page address, including the translatedphysical page address of the virtual page address. The second abnormalTLB access is a page fault, wherein the TLB information indicates thememory page specified by the virtual page address is not present in thesystem memory. The third abnormal TLB access is a memory protectionfault, wherein the TLB information indicates the access to the memorypage specified by the virtual page address constitutes a memoryprotection violation.

Stream_prefetch_priority_parameters 614 also include a TLB_miss_policyfield 806. TLB_miss_policy field 806 specifies the action memorysubsystem 104 takes in the event current_prefetch_address 324 of streamprefetch request 136 misses in a TLB of memory subsystem 104, which isan abnormal TLB access. In one embodiment, the value of TLB_miss_policyfield 806 is forwarded to the memory subsystem 104 in stream prefetchrequest 136. In one embodiment, the programmer may specify two possibleactions in response to a TLB miss. If the programmer specifies viaTLB_miss_policy field 806 a normal action in response to a TLB miss,then memory subsystem 104 services the stream prefetch TLB miss as itwould for other load or store TLB misses, which generates more trafficon processor bus 132, potentially consuming precious microprocessor 100resources which might otherwise be used for higher priority operations.However, if the programmer specifies an abort action in response to aTLB miss, then memory subsystem 104 aborts the stream prefetch request136 and does not prefetch the specified cache line into memory subsystem104, thereby not incurring the overhead associated with servicing theTLB miss. Hence, the TLB_miss_policy parameter 806 provides a means ofenabling the programmer to specify the priority of stream prefetchoperations relative to other operations in microprocessor 100. Theoperation of microprocessor 100 based on the TLB_miss_policy field 806is described in detail below with respect to FIG. 14.

Stream_prefetch_priority_parameters 614 also include a page_fault_policyfield 808. Page_fault_policy field 808 specifies the action memorysubsystem 104 takes in the event that a page of memory implicated bycurrent_prefetch_address 324 of stream prefetch request 136 is notpresent in memory, referred to as a page fault, which is an abnormal TLBaccess. In one embodiment, the value of page_fault_policy field 808 isforwarded to the memory subsystem 104 in stream prefetch request 136. Inone embodiment, the programmer may specify two possible actions inresponse to a page fault. If the programmer specifies viapage_fault_policy field 808 a normal action in response to a page fault,then memory subsystem 104 services the stream prefetch page fault as itwould for other load or store page faults, which typically results in arelatively huge latency of the operating system fetching the memory pagefrom mass storage in the computer system, such as a disk drive,potentially consuming precious microprocessor 100 resources which mightotherwise be used for higher priority operations. However, if theprogrammer specifies an abort action in response to a page fault, thenmemory subsystem 104 aborts the stream prefetch request 136 and does notprefetch the specified cache line into memory subsystem 104, thereby notincurring the overhead associated with servicing the page fault. Hence,the page_fault_policy parameter 808 provides a means of enabling theprogrammer to specify the priority of stream prefetch operationsrelative to other operations in microprocessor 100. The operation ofmicroprocessor 100 based on the page_fault_policy field 808 is describedin detail below with respect to FIG. 15.

Stream_prefetch_priority_parameters 614 also include aprotection_fault_policy field 812. Protection_fault_policy field 812specifies the action memory subsystem 104 takes in the event that thetask or process executing the stream prefetch instruction 138 does nothave permission to access the location in memory specified bycurrent_prefetch_address 324 of stream prefetch request 136, referred toas a protection fault, which is an abnormal TLB access. In oneembodiment, the value of protection_fault_policy field 812 is forwardedto the memory subsystem 104 in stream prefetch request 136. In oneembodiment, the programmer may specify two possible actions in responseto a protection fault. If the programmer specifies viaprotection_fault_policy field 812 a normal action in response to aprotection fault, then memory subsystem 104 services the stream prefetchprotection fault as it would for other load or store protection faults,which typically results in a relatively large latency of the operatingsystem invoking a protection fault service routine and potentiallyterminating the running program. However, if the programmer specifies anabort action in response to a protection fault, then memory subsystem104 aborts the stream prefetch request 136 without generating aprotection fault to the operating system and does not prefetch thespecified cache line into memory subsystem 104. Hence, theprotection_fault_policy parameter 812 provides a means of enabling theprogrammer to specify the priority of stream prefetch operationsrelative to other operations in microprocessor 100. Furthermore,protection_fault_policy parameter 812 advantageously provides a means ofenabling the programmer to specify that the data stream prefetch isallowed to be speculative in nature. For example, a program mightspecify in the stream prefetch instruction 138 a data stream to beprefetched that is larger than what is ultimately consumed by theprogram because the programmer was not sure of the run-time data streamsize at the time the stream prefetch instruction 138 is executed.However, the programmer still wants to take maximum advantage ofprefetching in the case the run-time data stream turns out to be large.Thus the programmer may optimistically specify a large data stream viathe stream descriptor 600 that runs over his task's valid memory space.In this case the programmer would not want to take a protection fault,which could result in the operating system killing the taskunnecessarily. Hence, the protection_fault_policy parameter 812advantageously allows the programmer more freedom in specifyingspeculative prefetch streams. The operation of microprocessor 100 basedon the protection_fault_policy field 812 is described in detail belowwith respect to FIG. 16.

Stream_prefetch_priority_parameters 614 also include astream_prefetch_priority field 814. Stream_prefetch_priority field 814specifies the priority to be given to processor bus 132 transactionsassociated with stream prefetch request 136 relative to other processorbus 132 transactions. In one embodiment, memory subsystem 104 usesstream_prefetch_priority field 814 in conjunction with a task priorityto determine how to schedule competing processor bus 132 transactionrequests. That is, the operating system loads a task priority intomicroprocessor 100 (such as in a system register in register file 124 orin a memory management unit of microprocessor 100) for each taskcurrently executing in microprocessor 100. When instructiondecode/dispatch unit 122 of FIG. 1 dispatches a load, store, or streamprefetch instruction, it issues the task priority of the task executingthe instruction along with the instruction for use by the load/storeunit 116, stream prefetch unit 118, memory subsystem 104, and businterface unit 102 to prioritize competing processor bus 132 transactionrequests. In one embodiment, stream prefetch request 136 includes thevalue of stream_prefetch_priority indicator 814 for use by memorysubsystem 104 so that memory subsystem 104 can communicate to businterface unit 102 the relative priority of a bus request issued bymemory subsystem 104 to bus interface unit 102 in response to the streamprefetch request 136, thereby enabling arbiter 142 of FIG. 1 to properlyprioritize the stream prefetch bus request relative to other busrequests in request queue 144 of FIG. 1. The stream_prefetch_priorityfield 814 advantageously enables a programmer to allocate the usage ofbandwidth on processor bus 132 to stream prefetches appropriately, whichmay be very helpful toward improving performance, since there may be anabundance or shortage of processor bus 132 bandwidth available atdifferent times due to other activity within microprocessor 100.

In one embodiment, stream_prefetch_priority field 814 may be one of fourpossible values, as follows. A value of 0 corresponds to a maximumpriority, and instructs microprocessor 100 to schedule the bus requestassociated with the stream prefetch request 136 for the requestingstream prefetch engine 202 in front of loads and stores from lowerpriority tasks, in front of stream prefetch requests from all otherstream prefetch engines 202, and behind loads and stores from higherpriority tasks. A value of 1 corresponds to a high priority, andinstructs microprocessor 100 to schedule the bus request associated withthe stream prefetch request 136 for the requesting stream prefetchengine 202 in front of stream prefetch requests from other mediumpriority stream prefetch engines 202, round-robin with otherhigh-priority stream prefetch engines 202, and behind loads and storesfrom all other tasks. A value of 2 corresponds to a medium priority, andinstructs microprocessor 100 to schedule the bus request associated withthe stream prefetch request 136 for the requesting stream prefetchengine 202 round-robin with stream prefetch requests from other mediumpriority stream prefetch engines 202, and behind loads and stores fromall other tasks. A value of 3 corresponds to a low priority, andinstructs microprocessor 100 to schedule the bus request associated withthe stream prefetch request 136 for the requesting stream prefetchengine 202 behind all stream prefetch requests from all other streamprefetch engines 202, and behind loads and stores from all other tasks.

Stream_prefetch_priority_parameters 614 also include aload/store_monitor_policy field 816. Load/store_monitor_policy field 816specifies whether stream prefetch engine 202 monitors load/store request134 for loads, stores, or both to determine whether a hit in the datastream has occurred. In one embodiment, a value of 0 instructs streamprefetch engine 202 to monitor loads; a value of 1 instructs streamprefetch engine 202 to monitor stores; a value of 2 instructs streamprefetch engine 202 to monitor both loads and stores.

Stream_prefetch_priority_parameters 614 also include a stream prefetchhysteresis field 818. Stream_prefetch_hysteresis field 818 specifies theminimum number of bytes to prefetch whenever stream prefetching isresumed, i.e., whenever stream prefetching is triggered by a load/storerequest 134 hit in the data stream within thestream_fetch-ahead_distance 612 of the current_prefetch_addr 324. Use ofthe stream_prefetch_hysteresis field 818 is described in detail belowwith respect to FIG. 13.

Stream_prefetch_priority_parameters 614 also include aspeculative_stream_hit_policy field 822. Speculative_stream_hit_policyfield 822 enables the programmer to specify whether the stream prefetchengine 202 should trigger prefetching of the data stream, as in decisionblock 1114 of FIG. 11, if the load or store that hit in the data streamis a speculative load or store instruction. In one embodiment, if thespeculative_stream_hit_policy field 822 is 0, then data streamprefetching is triggered, and if the speculative_stream_hit_policy field822 is 1, then data stream prefetching is not triggered. A speculativeload or store instruction is a load or store instruction that isspeculatively executed, i.e., is not guaranteed to complete. Typicallyan instruction is speculative because it was executed based on a branchprediction made by the microprocessor 100 that may later be discoveredto have been incorrect and require correction by flushing all of thespeculatively executed instructions.

Referring now to FIG. 9, a block diagram illustrating an example datastream template 904 specified by a stream descriptor 600 of FIG. 6according to the present invention is shown. FIG. 9 shows system memory902, which includes a stream template 904 within a subset of the systemmemory 902 address space. The stream template 904 is exploded in FIG. 9to show an example stream template 904 having nine stream blocks 906.One of the stream blocks is exploded to show that the length of a streamblock 906 is defined by the stream_block_size 608. For example, aprogrammer might specify the stream block 906 of FIG. 9 by thestream_block_size field 608 to be 120 bytes in length. The beginning ofthe data stream template 904 is shown marked by the stream_base 602 ofFIG. 6. Additionally, the end address of the data stream template 904 isindicated by the distance covered by the stream_length 604 of FIG. 6from the stream_base 602. FIG. 9 also illustrates thestream_block_stride 606 of FIG. 6 as the distance between the beginningof one stream block 906 and the beginning of the next stream block 906.FIG. 9 also illustrates an example stream_fetch-ahead_distance 612 ofFIG. 6 as three stream_block_strides 606 worth of bytes.

Referring now to FIG. 10, a block diagram illustrating conditions whichselectively trigger prefetching of a data stream with the streamtemplate example of FIG. 9 according to the present invention is shown.FIG. 10 shows two situations. In each of the situations, thecurrent_prefetch_addr 324 of FIG. 3 specifies an address within theseventh stream block 906. Consequently, the beginning of thestream_fetch-ahead_distance 612 is shown specifying an address withinthe fourth stream block 906, since the stream_fetch-ahead_distance 612in the example of FIG. 9 is three stream_block_strides 606. In situation#1, the current_stream_hit_addr 322 of FIG. 3 specifies an addresswithin the second stream block 906. Hence, thecurrent_fetch-ahead_distance 344 is six stream blocks 906, as shown.Consequently, the corresponding stream prefetch engine 202 of FIG. 2remains suspended since the current_fetch-ahead_distance 344 is greaterthan the stream_fetch-ahead_distance 612. In situation #2, thecurrent_stream_hit_addr 322 of FIG. 3 specifies an address within thefifth stream block 906. Hence, the current_fetch-ahead_distance 344 istwo stream blocks 906, as shown. Consequently, the corresponding streamprefetch engine 202 of FIG. 2 resumes prefetching of the specified datastream since the current_fetch-ahead_distance 344 is less than thestream_fetch-ahead_distance 612, i.e., prefetching of the data stream istriggered by the detection of current_stream_hit_addr 322 within thestream_fetch-ahead_distance 612. This operation is described in detailwith respect to FIGS. 11 through 13 below.

Referring now to FIG. 11, a flowchart illustrating stream prefetchingaccording to the present invention is shown. Flow begins a block 1102.

At block 1102, instruction decode/dispatch unit 122 decodes anddispatches a stream prefetch instruction 138 to stream prefetch unit 118of FIG. 1. Flow proceeds to block 1104.

At block 1104, stream engine allocator 204 of FIG. 2 receives the streamprefetch instruction 138, allocates one of the stream prefetch engines202 of FIG. 2, and initializes the allocated stream prefetch engine 202with the stream descriptor 600 of stream prefetch instruction 138. Inparticular, the stream descriptor 600 is loaded into stream descriptorregisters 362 of FIG. 3. Additionally, the stream_ID of the allocatedstream prefetch engine 202 is returned to a predetermined generalpurpose register of register file 124 of FIG. 1. If none of the streamprefetch engines 202 are free for allocation, then the stream prefetchinstruction 138 returns a 0 stream_ID value; otherwise, the streamprefetch instruction 138 returns a stream_ID value identifying theallocated stream prefetch engine 202. Flow proceeds to block 1106.

At block 1106, control logic 334 of FIG. 3 loads the stream_base 602from stream_base register 302 into current_prefetch_addr register 324and current_stream_block_start register 326. Flow proceeds to block1108.

At block 1108, control logic 334 begins prefetching the data stream intomemory subsystem 104 as specified by the stream descriptor 600 until thecurrent_prefetch_addr register 324 exceeds the stream_base register 302value by the stream_fetch-ahead_distance register 312 value. That is,control logic 334 continuously and asynchronously issues stream prefetchrequests 136 to memory subsystem 104 following the data stream templatedescribed in the stream descriptor 600 and updating thecurrent_prefetch_addr register 324 until the current_prefetch_addrregister 324 exceeds the stream_base register 302 value by thestream_fetch-ahead_distance register 312 value. Flow proceeds to block1112.

At block 1112, control logic 334 suspends prefetching of the datastream. That is, control logic 334 stops issuing stream prefetch request136 to memory subsystem 104 and begins monitoring load/store requests134. Flow proceeds to decision block 1114.

At decision block 1114, control logic 334 determines whether aload/store request 134 has been issued and if so, whether the load/storerequest 134 hits in the data stream. That is, control logic 334 examineshit_in_stream signal 342 to determine whether the load/store request 134hit in the data stream. Additionally, control logic 334 examinesload/store request 134 to determine whether the load/store request 134was of the type being monitored. That is, although FIG. 11 states adetermination is made whether a load request hits in the data stream, itis understood that control logic 334 determines whether the load/storerequest 134 was a load, or a store, or either, depending upon themonitoring policy. In one embodiment, control logic 334 monitors onlyload requests. In one embodiment, control logic 334 monitors only storerequests. In one embodiment, control logic 334 monitors both load andstore requests. In one embodiment, control logic 334 monitors requestsbased on the type specified by the programmer inload/store_monitor_policy field 816 of FIG. 8 of stream descriptor 600stored in stream_prefetch_priority_parameters register 314, as describedabove with respect to FIG. 8. Additionally, in one embodiment, controllogic 334 examines load/store request 134 to determine whether theload/store request 134 was speculative and whether thespeculative_stream_hit_policy field 822 specifies a policy of triggeringon speculative stream hits. If a load/store request 134 of a monitoredtype hits in the data stream, flow proceeds to block 1116; otherwise,flow returns to decision block 1114 to monitor the next load/storerequest 134.

At block 1116, control logic 334 updates current_stream_hit_addr 322with the address of load/store request 134 that hit in the data streamas determined at decision block 1114. Flow proceeds to block 1118.

At block 1118, subtractor 352 calculates thecurrent_fetch-ahead_distance 344 of FIG. 3 by subtracting thecurrent_stream_hit_addr 322 from the current_prefetch_addr 324. Flowproceeds to decision block 1122.

At decision block 1122, control logic 334 determines whether thecurrent_fetch-ahead_distance 344 is less than thestream_fetch-ahead_distance 612 stored in stream_fetch-ahead distanceregister 312. If so, flow proceeds to decision block 1124; otherwise,flow returns to decision block 1114 to monitor the next load/storerequest 134.

At decision block 1124, control logic 334 determines whether prefetchingof the data stream is currently suspended. In one embodiment, controllogic 334 maintains state that specifies whether stream prefetching iscurrently suspended or resumed. If prefetching of the data stream issuspended, flow proceeds to block 1126; otherwise, flow returns todecision block 1114 to monitor the next load/store request 134, sincestream prefetching is already in progress.

At block 1126, control logic 334 resumes prefetching the data streaminto memory subsystem 104 as specified by the stream descriptor 600until the current_prefetch_addr register 324 exceeds thecurrent_stream_hit_addr register 322 value by thestream_fetch-ahead_distance register 312 value. Block 1126 is describedin more detail with respect to FIGS. 12 and 13 below. Flow proceeds toblock 1128.

At block 1128, control logic 334 suspends prefetching of the datastream. Flow returns to decision block 1114 to monitor the nextload/store request 134. If the programmer specifies a stream_length 604value of 0, then the data stream is unbounded, and the stream prefetchengine 202 continues synchronously prefetching the data stream as shownin FIG. 11 until halted by execution of a halt stream 700 instruction;however, if the programmer specifies a non-zero stream_length 604 valuethen flow ends once the current_prefetch_addr 324 reaches the end of thedata stream. Because the stream prefetch engine 202 does notcontinuously generate stream prefetch requests 136, but insteadadvantageously only generates stream prefetch requests 136 synchronizedwith program load instruction execution (or stores or both) based on thespecified stream_fetch-ahead distance 612 as described herein,specifying an unbounded stream does not have the problems suffered bytraditional stream prefetch solutions of wasting memory bandwidth orprematurely evicting more useful data from the cache, thereby pollutingthe cache with unneeded data.

Referring now to FIG. 12, a flowchart illustrating in detail block 1126of FIG. 11 according to the present invention is shown. Flow proceeds toblock 1202 from decision block 1124 of FIG. 11.

At block 1202, control logic 334 generates a stream prefetch request 136to memory subsystem 104 to prefetch a cache line containingcurrent_prefetch_addr 324 into memory subsystem 104; memory subsystem104 generates a request to bus interface unit 102 to prefetch the cacheline into memory subsystem 104; and bus interface unit 102 generates atransaction on processor bus 132 to prefetch the cache line into memorysubsystem 104. However, if current_prefetch_addr 324 hits in memorysubsystem 104, then memory subsystem 104 does not generate the requestto bus interface unit 102 to prefetch the cache line. Flow proceeds toblock 1204.

At block 1204, control logic 334 increments the current_prefetch_addrregister 324 by the cache line size. Flow proceeds to decision block1206.

At decision block 1206, control logic 334 determines whether the currentstream block has been prefetched. If so, flow proceeds to block 1208;otherwise, flow returns to block 1202.

At block 1208, control logic 334 updates current_prefetch_addr register324 with the sum of the contents of current_stream_block_start register326 and stream_block_stride register 306. Flow proceeds to block 1212.

At block 1212, control logic 334 updates current_stream_block_startregister 326 with the value in current_prefetch_addr register 324. Flowproceeds to block 1214.

At block 1214, subtractor 352 calculates thecurrent_fetch-ahead_distance 344 of FIG. 3 by subtracting thecurrent_stream_hit_addr 322 from the current_prefetch_addr 324. Flowproceeds to decision block 1216.

At decision block 1216, control logic 334 determines whether thecurrent_fetch-ahead_distance 344 is less than thestream_fetch-ahead_distance 612 stored in stream_fetch-ahead distanceregister 312. If so, flow proceeds to block 1128 of FIG. 11; otherwise,flow returns to block 1202.

Referring now to FIG. 13, a flowchart illustrating in detail block 1126of FIG. 11 according to an alternate embodiment of the present inventionis shown. The embodiment of FIG. 13 employs thestream_prefetch_hysteresis parameter 818 of FIG. 8. FIG. 13 is similarto FIG. 12 and like numbered blocks are the same. However, FIG. 13 alsoincludes three additional blocks—1302, 1304, and decision block1306—which are described below. Flow proceeds to block 1302 fromdecision block 1124 of FIG. 11.

At block 1302, control logic 334 initializes a byte count to 0. Flowproceeds to block 1202.

At block 1202, control logic 334 generates a stream prefetch request 136to memory subsystem 104 to prefetch a cache line containingcurrent_prefetch_addr 324 into memory subsystem 104; memory subsystem104 generates a request to bus interface unit 102 to prefetch the cacheline into memory subsystem 104; and bus interface unit 102 generates atransaction on processor bus 132 to prefetch the cache line into memorysubsystem 104. However, if current_prefetch_addr 324 hits in thespecified cache of memory subsystem 104, then memory subsystem 104 doesnot generate the request to bus interface unit 102 to prefetch the cacheline. Flow proceeds to block 1204.

At block 1204, control logic 334 increments the current_prefetch_addrregister 324 by the cache line size. Flow proceeds to block 1304.

At block 1304, control logic 334 increments the byte count by the sizeof a cache line. Flow proceeds to decision block 1206.

At decision block 1206, control logic 334 determines whether the currentstream block has been prefetched. If so, flow proceeds to block 1208;otherwise, flow returns to block 1202.

At block 1208, control logic 334 updates current_prefetch_addr register324 with the sum of the contents of current_stream_block_start register326 and stream_block_stride register 306. Flow proceeds to block 1212.

At block 1212, control logic 334 updates current_stream_block_startregister 326 with the value in current_prefetch_addr register 324. Flowproceeds to block 1214.

At block 1214, subtractor 352 calculates thecurrent_fetch-ahead_distance 344 of FIG. 3 by subtracting thecurrent_stream_hit_addr 322 from the current_prefetch_addr 324. Flowproceeds to decision block 1216.

At decision block 1216, control logic 334 determines whether thecurrent_fetch-ahead_distance 344 is less than thestream_fetch-ahead_distance 612 stored in stream_fetch-ahead distanceregister 312. If so, flow proceeds to decision block 1306; otherwise,flow returns to block 1202.

At decision block 1306, control logic 334 determines whether the bytecount is greater than or equal to the stream_prefetch_hysteresisparameter 818 stored in stream_prefetch_priority_parameters register314. If so, flow proceeds to block 1128 of FIG. 11; otherwise, flowreturns to block 1202.

As may be seen from FIG. 13, the stream_prefetch_hysteresis field 818enables the programmer to specify a minimum amount of the data stream toprefetch in a chunk, which is advantageous because it potentiallyenables bus interface unit 102 to combine multiple smaller streamprefetch requests 136 into one or more larger bus transaction requestson processor bus 132, thereby more efficiently using processor bus 132and system memory bandwidth.

The prefetching of the data stream that is synchronized with programexecution of loads, stores, or both, advantageously avoids some of thedisadvantages of a stream prefetch instruction that does not have anyhardware means of synchronizing with program execution. In particular, atraditional stream prefetch solution can easily get too far ahead of,i.e., overrun, the program execution, causing prefetched portions of thedata stream to get evicted from the cache or become stale before theprogram has a chance to consume it. This phenomenon not only destroysthe effectiveness of the stream prefetch, but potentially reducesperformance rather than improving it by wasting memory bandwidth andprematurely evicting more useful data from the cache, thereby pollutingthe cache with unneeded data. However, as may be seen from FIGS. 10through 13, stream hit detector 332 of FIG. 3 advantageously detectsloads (or stores or both, depending on the monitoring policy, which inone embodiment is defined by the value of load/store_monitor_policyfield 816 discussed above) that hit anywhere within the specified datastream template, and control logic 334 of FIG. 3 uses the hitinformation to determine whether the hit is within thestream_fetch-ahead_distance 612. The data stream hit detection is usedto synchronize suspension and resumption of data stream prefetching asdescribed herein.

Referring now to FIG. 14, a flowchart illustrating operation ofmicroprocessor 100 in response to an abnormal TLB access, in particulara TLB miss in memory subsystem 104 of a stream prefetch request 136 ofFIG. 1 according to the present invention is shown. Flow begins at block1402.

At block 1402, memory subsystem 104 detects a miss in one of the TLBs ofmemory subsystem 104. Flow proceeds to decision block 1404.

At decision block 1404, memory subsystem 104 determines whether the TLBmiss detected in block 1402 was caused by a stream prefetch request 136,or whether the TLB miss was caused by a load/store request 134. If by astream prefetch request 136, flow proceeds to decision block 1412;otherwise, the TLB miss was caused by a load/store request 134, and flowproceeds to block 1406.

At block 1406, memory subsystem 104 services the TLB miss by fetchingfrom the system memory the missing TLB information and updating the TLBtherewith. In the embodiment having a unified TLB, memory subsystem 104updates the unified TLB. In the embodiment of FIGS. 17 through 19,memory subsystem 104 updates load/store TLB 1704. Flow proceeds to block1408.

At block 1408, memory subsystem 104 completes the load/store request 134since the TLB information is now in the TLB. Flow ends at block 1408.

At decision block 1412, memory subsystem 104 determines what the policyis for a TLB miss generated by a stream prefetch request 136. In theembodiment of FIG. 18 or an embodiment in which memory subsystem 104employs a unified TLB for load/store requests 134 and stream prefetchrequests 136, memory subsystem 104 determines the TLB miss policy forstream prefetch requests by examining TLB_miss_policy parameter 806 ofFIG. 8 specified in stream descriptor 600 and stored instream_prefetch_priority_parameters register 314 and forwarded to memorysubsystem 104 in stream prefetch request 136 to determine whether thepolicy is to abort the stream prefetch request 136 or to handle thestream prefetch request 136 normally like any other load/store request134. In the embodiments of FIG. 17 and FIG. 19, memory subsystem 104employs a normal TLB miss policy, since a load/store request 134 missingin load/store TLB 1704 will not populate the relevant stream prefetchTLB 1702 or 1902A-D. If the TLB miss policy is an abort policy, thenflow proceeds to block 1418; otherwise, flow proceeds to block 1414.

At block 1414, memory subsystem 104 services the TLB miss by fetchingfrom the system memory the missing TLB information and updating the TLBtherewith. In the embodiment having a unified TLB, memory subsystem 104updates the unified TLB. In the embodiment of FIGS. 17 through 19,memory subsystem 104 updates the relevant stream prefetch TLB 1702 or1902A-D. Flow proceeds to block 1416.

At block 1416, memory subsystem 104 completes the stream prefetchrequest 136 since the TLB information is now in the TLB. Flow ends atblock 1416.

At block 1418, memory subsystem 104 aborts the stream prefetch request136 without updating the TLB with the missing information. Flow proceedsto block 1422. The dotted line from block 1418 to block 1422 in FIG. 14denotes that block 1422 follows block 1418, but is asynchronous toaborting the stream prefetch request 136.

At block 1422, a load/store unit 116 subsequently generates a load/storerequest 134 to the same memory page implicated by the aborted streamprefetch request 136, and the load/store request 134 also misses in theTLB. Flow proceeds to block 1424.

At block 1424, memory subsystem 104 services the TLB miss caused by theload/store request 134 by fetching from the system memory the missingTLB information and updating the TLB therewith. In the embodiment havinga unified TLB, memory subsystem 104 updates the unified TLB. In theembodiment of FIGS. 17 and 19, block 1424 is not relevant. In theembodiment of FIG. 18, memory subsystem 104 updates joint TLB 1802. Flowproceeds to block 1426.

At block 1426, memory subsystem 104 completes the load/store request 134since the TLB information is now in the TLB. Flow proceeds to block1428.

At block 1428, stream prefetch engine 202 detects that a subsequentload/store request 134 hits in the data stream that is within thestream_fetch-ahead_distance 612, as determined at blocks 1114 through1124 of FIG. 11, causing stream prefetch engine 202 to resumeprefetching, according to block 1126 of FIG. 11, and in particular, togenerate a stream prefetch request 136 to the same current_prefetch_addr324 that missed in the TLB according to block 1402, but which now hitsin the TLB since the TLB was previously populated with the missing TLBinformation according to block 1424. Flow ends at block 1428.

Referring now to FIG. 15, a flowchart illustrating operation ofmicroprocessor 100 in response to an abnormal TLB access, in particulara page fault caused by a stream prefetch request 136 of FIG. 1 accordingto the present invention is shown. It is noted that typically a TLB missoccurs prior to a page fault. Flow begins at block 1502.

At block 1502, memory subsystem 104 detects a condition in which arequested memory page is not present in system memory. Flow proceeds todecision block 1504.

At decision block 1504, memory subsystem 104 determines whether the pagefault detected in block 1502 was caused by a stream prefetch request136, or whether the page fault was caused by a load/store request 134.If by a stream prefetch request 136, flow proceeds to decision block1512; otherwise, the page fault was caused by a load/store request 134,and flow proceeds to block 1506.

At block 1506, microprocessor 100 notifies the operating system of thepage fault, and the operating system fetches the missing page from amass storage device into the system memory. Flow proceeds to block 1508.

At block 1508, memory subsystem 104 completes the load/store request 134since the page is now in the system memory. Flow ends at block 1508.

At decision block 1512, memory subsystem 104 determines what the policyis for a page fault generated by a stream prefetch request 136. In oneembodiment, the page fault policy is always to abort the stream prefetchrequest 136. In another embodiment, memory subsystem 104 determines thepage fault policy for stream prefetch requests by examiningpage_fault_policy parameter 808 of FIG. 8 specified in stream descriptor600 and stored in stream_prefetch_priority_parameters register 314 andforwarded to memory subsystem 104 in stream prefetch request 136 todetermine whether the policy is to abort the stream prefetch request 136or to handle the stream prefetch request 136 normally like any otherload/store request 134. If the page fault policy is to abort, then flowproceeds to block 1518; otherwise, flow proceeds to block 1514.

At block 1514, microprocessor 100 notifies the operating system of thepage fault, and the operating system fetches the missing page from amass storage device into the system memory. Flow proceeds to block 1516.

At block 1516, memory subsystem 104 completes the stream prefetchrequest 136 since the missing page is now in the system memory. Flowends at block 1516.

At block 1518, memory subsystem 104 aborts the stream prefetch request136 without notifying the operating system of the page fault. Flowproceeds to block 1522. The dotted line from block 1518 to block 1522 inFIG. 14 denotes that block 1522 follows block 1518, but is asynchronousto aborting the stream prefetch request 136.

At block 1522, a load/store unit 116 subsequently generates a load/storerequest 134 to the same memory page implicated by the aborted streamprefetch request 136, and the load/store request 134 also generates apage fault. Flow proceeds to block 1524.

At block 1524, microprocessor 100 notifies the operating system of thepage fault, and the operating system fetches the missing page from amass storage device into the system memory. Flow proceeds to block 1526.

At block 1526, memory subsystem 104 completes the load/store request 134since the missing page is now in the system memory. Flow proceeds toblock 1528.

At block 1528, stream prefetch engine 202 detects that a subsequentload/store request 134 hits in the data stream that is within thestream_fetch-ahead_distance 612, as determined at blocks 1114 through1124 of FIG. 11, causing stream prefetch engine 202 to resumeprefetching, according to block 1126 of FIG. 11, and in particular, togenerate a stream prefetch request 136 to the same current_prefetch_addr324 that generated the page fault according to block 1502, but for whichthe implicated page is now in system memory since the missing page wasbrought into the system memory according to block 1524. Flow ends atblock 1528.

Referring now to FIG. 16, a flowchart illustrating operation ofmicroprocessor 100 in response to an abnormal TLB access, in particulara protection fault caused by a stream prefetch request 136 of FIG. 1according to the present invention is shown. Flow begins at block 1602.

At block 1602, memory subsystem 104 detects a request that specifies alocation in system memory which violates the memory protection policy.Flow proceeds to decision block 1604.

At decision block 1604, memory subsystem 104 determines whether theprotection fault detected in block 1602 was caused by a stream prefetchrequest 136, or whether the protection fault was caused by a load/storerequest 134. If by a stream prefetch request 136, flow proceeds todecision block 1612; otherwise, the protection fault was caused by aload/store request 134, and flow proceeds to block 1606.

At block 1606, microprocessor 100 notifies the operating system that amemory protection violation has occurred. Flow ends at block 1606.

At decision block 1612, memory subsystem 104 determines what the policyis for a protection fault generated by a stream prefetch request 136. Inone embodiment, the protection fault miss policy is always to abort thestream prefetch request 136. In another embodiment, memory subsystem 104determines the protection fault policy for stream prefetch requests byexamining protection_fault_policy parameter 812 of FIG. 8 specified instream descriptor 600 and stored in stream_prefetch_priority_parametersregister 314 and forwarded to memory subsystem 104 in stream prefetchrequest 136 to determine whether the policy is to abort the streamprefetch request 136 or to handle the stream prefetch request 136normally like any other load/store request 134. If the protection faultpolicy is to abort, then flow proceeds to block 1618; otherwise, flowproceeds to block 1614.

At block 1614, microprocessor 100 notifies the operating system that amemory protection violation has occurred. Flow ends at block 1614.

At block 1618, memory subsystem 104 aborts the stream prefetch request136 without notifying the operating system of the memory protectionviolation. Flow ends at block 1618.

Referring now to FIG. 17, a block diagram of portions of memorysubsystem 104 of FIG. 1 having a separate stream prefetch TLB accordingto the present invention is shown.

Memory subsystem 104 includes a load/store TLB 1704, coupled to receiveload/store request 134, for caching virtual page addresses of load/storerequests 134 and TLB information associated therewith. Load/storerequest 134 includes a virtual page address, which is looked up byload/store TLB 1704 and used to select an entry of TLB information inload/store TLB 1704. If the virtual page address of load/store request134 misses in load/store TLB 1704, then load/store TLB 1704 outputs atrue value on a miss signal 1744, which is provided to control logic1706 of memory subsystem 104. If the virtual page address hits, thenload/store TLB 1704 outputs a false value on a miss signal 1744, andprovides the selected TLB information on TLB information signal 1714.Additionally, control logic 1706 generates an update signal 1742 toupdate load/store TLB 1704 with new TLB information as necessary.

The TLB information 1714 stored in load/store TLB 1704 includes addresstranslation information, such as a translated physical page address ofthe virtual page address, and an indication of whether the pagespecified by the virtual page address is present in the system memory,which is used to detect page faults. The TLB information 1714 alsoincludes memory protection information about the memory protectionpolicy for the specified page, which is used to detect protectionfaults. The TLB information 1714 also includes attribute bits specifyingthe bus transaction priority for loads and stores of the specifiedmemory page, such as discussed above with respect to FIG. 6.

Memory subsystem 104 also includes a multiplexer 1712. Multiplexer 1712shown in the embodiment of FIG. 17 comprises a four-input multiplexerfor receiving stream prefetch requests 136A-D of FIG. 2. Control logic1706 arbitrates between the prefetch requests 136A-D for access to astream prefetch TLB 1702 and generates a control signal 1738 provided tomultiplexer 1712 to select one of the inputs for provision on an output1718.

Memory subsystem 104 also includes stream prefetch TLB 1702, coupled toreceive multiplexer 1712 output 1718. Stream prefetch TLB 1702 cachesvirtual page addresses of stream prefetch requests 136 and TLBinformation associated therewith. Stream prefetch request 136 includes avirtual page address, which is looked up by stream prefetch TLB 1702 andused to select an entry of TLB information in stream prefetch TLB 1702.If the virtual page address of stream prefetch request 136 misses instream prefetch TLB 1702, then stream prefetch TLB 1702 outputs a truevalue on a miss signal 1724, which is provided to control logic 1706. Ifthe virtual page address hits, then stream prefetch TLB 1702 outputs afalse value on a miss signal 1724, and provides the selected TLBinformation on TLB information signal 1716, which is similar to TLBinformation 1714, except TLB information 1716 stores information relatedto data stream prefetches. Additionally, control logic 1706 generates anupdate signal 1722 to update stream prefetch TLB 1702 with new TLBinformation as necessary.

Memory subsystem 104 also includes a two-input multiplexer 1708.Multiplexer 1708 receives TLB information 1714 on one input and TLBinformation 1716 on the other input. Multiplexer 1708 selects one of theinputs for output as TLB information 1726 for provision to relevantparts of memory subsystem 104, such as tag comparators, based on acontrol signal 1736 generated by control logic 1706, and for provisionto control logic 1706.

Referring now to FIG. 18, a block diagram of portions of memorysubsystem 104 of FIG. 1 having a separate stream prefetch TLB accordingto an alternate embodiment of the present invention is shown. FIG. 18 issimilar to FIG. 17, and like numbered elements are the same. FIG. 18includes control logic 1806, which is very similar to, but slightlydifferent from control logic 1706 of FIG. 17. The differences aredescribed below. In addition to the elements of FIG. 17, FIG. 18 alsoincludes a two-input multiplexer 1814 and a joint TLB 1802 that backsload/store TLB 1704 and stream prefetch TLB 1702. In one embodiment,joint TLB 1802 is a victim cache for load/store TLB 1704 and streamprefetch TLB 1702. An embodiment is also contemplated in which joint TLB1802 also backs L1 instruction cache 156 of FIG. 1.

Multiplexer 1814 receives load/store request 134 on one input andreceives the output 1718 of multiplexer 1712 on the other input.Multiplexer 1814 selects one of the inputs to provide on an output 1828based on a control signal 1804 generated by control logic 1806.

Joint TLB 1802 functions similarly to load/store TLB 1704 and streamprefetch TLB 1702, but with respect to both load/store requests 134 andstream prefetch requests 136, based on receiving signal 1828 as aninput. That is, joint TLB 1802 outputs TLB information 1826 similar toTLB information 1714 and 1716, outputs miss signal 1824 similar to misssignals 1724 and 1744, and receives update information 1822 similar toupdate information 1722 and 1742.

A three-input multiplexer 1808 of FIG. 18 replaces multiplexer 1708 ofFIG. 17. Multiplexer 1808 receives joint TLB 1802 TLB information 1826in addition to TLB information 1714 and TLB information 1716.Multiplexer 1808 selects one of the three inputs for output as TLBinformation 1726 for provision to relevant parts of memory subsystem104, such as tag comparators, based on a control signal 1836 generatedby control logic 1806, and for provision to control logic 1806.

Referring now to FIG. 19, a block diagram of portions of memorysubsystem 104 of FIG. 1 having a separate stream prefetch TLB accordingto an alternate embodiment of the present invention is shown. FIG. 19 issimilar to FIG. 17, except that a separate stream prefetch TLB isprovided for each stream prefetch engine 202A-D of FIG. 2. Memorysubsystem 104 includes a load/store TLB 1704 similar to that of FIG. 17.

Memory subsystem 104 also includes four stream prefetch TLBs 1902A-Dthat function similar to stream prefetch TLB 1702 of FIG. 17 withrespect to respective stream prefetch requests 136A-D generated bystream prefetch engines 202A-D, respectively. Stream prefetch TLBs1902A-D receive stream prefetch requests 136A-D, respectively, andgenerate TLB information 1716A-D and miss signals 1724A-D, respectively,and receive update information 1722A-D, respectively.

Memory subsystem 104 also includes a five-input multiplexer 1908.Multiplexer 1908 receives TLB information 1714 on one input and TLBinformation 1716A-D on the other four inputs. Multiplexer 1908 selectsone of the five inputs for output as TLB information 1726 for provisionto relevant parts of memory subsystem 104, such as tag comparators,based on a control signal 1936 generated by control logic 1906, and forprovision to control logic 1906.

As may be seen from FIGS. 17 through 19, the separate stream prefetchTLB 1702 of memory subsystem 104 advantageously avoids the undesirableresult of having stream prefetch operations pollute a unified TLB withrespect to loads and stores, which are typically higher priority.Conversely, TLB information for stream prefetch entries is less likelyto be evicted from a separate stream prefetch TLB, since they will notbe evicted by loads or stores. It has been observed that data streamaccesses are typically highly sequential in nature, rather than random.Furthermore, the latency associated with a TLB miss generated by astream prefetch request 136 may be absorbed by the effect of maintainingthe stream_fetch-ahead_distance 612, according to the present invention,while still avoiding polluting the load/store TLB 1704. Hence,advantageously a stream prefetch TLB 1702 may be very small. In oneembodiment of FIG. 19, each of the stream prefetch TLBs 1702 comprises asingle entry. In one embodiment of FIG. 19, each of the stream prefetchTLBs 1702 comprises two entries. Additionally, providing a separate TLBfor each stream prefetch engine 202, as in the embodiment of FIG. 19,avoids the undesirable result of having stream prefetch operations fromdisparate stream prefetch engines 202 pollute the stream prefetch TLB1702 of FIG. 17. An embodiment is also contemplated which combines thestream prefetch TLB per stream prefetch engine feature of FIG. 19 andthe joint TLB feature of FIG. 18.

Referring now to FIG. 20, a flowchart illustrating operation of streamhit detector 332 of FIG. 3 according to the present invention is shown.Flow begins at decision block 2002.

At decision block 2002, stream hit detector 332 determines whether thememory address of load/store request 134 of FIG. 1 is less than thestream_base 302 of FIG. 3. If so, flow proceeds to block 2004;otherwise, flow proceeds to decision block 2006.

At block 2004, stream hit detector 332 generates a false value onhit_in_stream signal 342 of FIG. 3. Flow ends at block 2004.

At decision block 2006, stream hit detector 332 determines whether thememory address of load/store request 134 of FIG. 1 is greater than orequal to the sum of the stream_base 302 and the stream_length 304 ofFIG. 3. If so, flow proceeds to block 2008; otherwise, flow proceeds todecision block 2012.

At block 2008, stream hit detector 332 generates a false value onhit_in_stream signal 342. Flow ends at block 2008.

At decision block 2012, stream hit detector 332 determines whether thedifference between the memory address of load/store request 134 and thestream_base 302, modulo the stream_block_stride 306 of FIG. 3, is lessthan the stream_block_size 308 of FIG. 3. If so, flow proceeds to block2016; otherwise, flow proceeds to decision block 2014.

At block 2014, stream hit detector 332 generates a false value onhit_in_stream signal 342. Flow ends at block 2014.

At block 2016, stream hit detector 332 generates a true value onhit_in_stream signal 342. Flow ends at block 2016.

In one embodiment, stream hit detector 332 approximates detection ofhits in the data stream. That is, rather than determining whether thememory address of load/store request 134 hits exactly within the datastream template specified by the stream descriptor 600, stream hitdetector 332 rounds to a reasonable size power of two, such as the cacheline size, to detect hits. Approximating detection of data stream hitssimplifies the logic required by stream hit detector 332, particularlythe logic required to perform the required modulo arithmetic.Advantageously, a small number of false hits in the data stream is notsignificantly detrimental since they would not constitute a largeadditional consumption of memory bandwidth or cache pollution.Furthermore, a small number of false misses in the data stream would notsignificantly undermine the goal of reducing memory fetch latency sinceit is highly likely that a subsequent load in the data stream willtrigger prefetching, and the additional time can easily be absorbed bythe stream_fetch-ahead distance 612.

Referring now to FIG. 21, a block diagram of stream hit detector 332 ofFIG. 3 according to one embodiment of the present invention is shown.

Stream hit detector 332 includes a request queue 2158 that buffers aplurality of load/store requests 134 of FIG. 1. When a load/storerequest 134 is generated, request queue 2158 loads the request 134 in afirst-in-first-out manner. Request queue 2158 outputs a load/storeaddress 2154 included in the oldest load/store request 134 storedtherein. Request queue 2158 generates a true value on an empty signal2162, which is provided to control logic 2116, whenever it is empty;otherwise, request queue 2158 generates a false value on empty signal2162. Control logic 2116 generates a true value on a shift signal 2164when it has finished determining whether the oldest load/store request134 in request queue 2158 hits in the data stream specified by streamdescriptor registers 362, in response to which request queue 2158 shiftsout the oldest load/store request 134. If the load/store request 134hits in the data stream, then it is loaded into current_stream_hit_addrregister 322 of FIG. 3. Request queue 2158 serves to reduce thelikelihood that a hit in the data stream template is not detected in theevent that load/store requests 134 are generated at a faster rate thanstream hit detector 332 can detect stream hits. In one embodiment,request queue 2158 is not included.

Stream hit detector 332 includes a first comparator 2102 that comparesload/store request address 2154 with the stream_base register 302 value.Comparator 2102 generates a true value on below_stream signal 2132,which is provided to control logic 2116, if load/store request address2154 is less than stream_base 302; otherwise, comparator 2102 generatesa false value on below_stream signal 2132.

Stream hit detector 332 also includes an adder 2122 that addsstream_base 302 and stream_length 304 to generate a stream_end signal2134.

Stream hit detector 332 also includes a second comparator 2104 thatcompares load/store address 2154 with stream_end 2134. Comparator 2104generates a true value on above_stream signal 2136, which is provided tocontrol logic 2116, if load/store address 2154 is greater than or equalto stream_end 2134; otherwise, comparator 2102 generates a false valueon above_stream signal 2136.

Stream hit detector 332 also includes a subtractor 2106 that subtractsstream_base 302 from load/store request address 2154 to generate anoffset_from_stream_base signal 2138.

Stream hit detector 332 also includes a modulo circuit 2114 thatperforms a modulo operation on offset_from_stream_base signal 2138,using the stream_block_stride 306 as the modulus, to generate amodular_offset_from_stream_base signal 2136.

Stream hit detector 332 also includes a third comparator 2108 thatcompares the modular_offset_from_stream_base 2136 with stream_block_size308 and generates a true value on within_block signal 2142 ifmodular_offset_from_stream_base 2136 is less than stream_block_size 308and generates a false value otherwise.

Control logic 2116 generates a true value on hit_in_stream signal 342 ofFIG. 3 if above_stream signal 2136 and below_stream signal 2132 are bothfalse and within_block signal 2142 is true.

Referring now to FIG. 22, a flowchart illustrating in detail block 1202of FIG. 12 according to the present invention is shown. Flow begins atdecision block 2202.

At decision block 2202, memory subsystem 104 of FIG. 1 determineswhether the value of cache_level indicator 802 of FIG. 8 of streamprefetch request 136 of FIG. 1 equals a value of 1. If so flow proceedsto block 2204; otherwise, flow proceeds to decision block 2206.

At block 2204, memory subsystem 104 generates a request to bus interfaceunit 102 of FIG. 1 to prefetch the cache line containing the locationspecified by current_prefetch_address 324 of FIG. 3 in stream prefetchrequest 136 from the system memory into L1 data cache 158 of FIG. 1.Flow ends at block 2204.

At decision block 2206, memory subsystem 104 determines whether thevalue of cache_level indicator 802 of stream prefetch request 136 equalsa value of 2. If so flow proceeds to block 2208; otherwise, flowproceeds to decision block 2212.

At block 2208, memory subsystem 104 generates a request to bus interfaceunit 102 to prefetch the cache line containing the location specified bycurrent_prefetch_address 324 in stream prefetch request 136 from thesystem memory into L2 cache 154 of FIG. 1. Flow ends at block 2208.

At decision block 2212, memory subsystem 104 determines whether thevalue of cache_level indicator 802 of stream prefetch request 136 equalsa value of 3. If so flow proceeds to block 2214; otherwise, flowproceeds to decision block 2216.

At block 2214, memory subsystem 104 generates a request to bus interfaceunit 102 to prefetch the cache line containing the location specified bycurrent_prefetch_address 324 in stream prefetch request 136 from thesystem memory into L3 cache 152 of FIG. 1. Flow ends at block 2214.

At decision block 2216, memory subsystem 104 determines whether thevalue of cache_level indicator 802 of stream prefetch request 136 equalsa value of 4. If so flow proceeds to block 2218; otherwise, flowproceeds to decision block 2222.

At block 2218, memory subsystem 104 generates a request to bus interfaceunit 102 to prefetch the cache line containing the location specified bycurrent_prefetch_address 324 in stream prefetch request 136 from thesystem memory into a prefetch buffer of memory subsystem 104 not shownin the embodiment of FIG. 1. Flow ends at block 2218.

At decision block 2222, memory subsystem 104 determines whether thevalue of cache_level indicator 802 of stream prefetch request 136 equalsa value of 5. If so flow proceeds to block 2224; otherwise, flowproceeds to decision block 2226.

At block 2224, memory subsystem 104 generates a request to bus interfaceunit 102 to prefetch the cache line containing the location specified bycurrent_prefetch_address 324 in stream prefetch request 136 from thesystem memory into L1 instruction cache 156 of FIG. 1. Flow ends atblock 2224.

At decision block 2226, memory subsystem 104 determines whether thevalue of cache_level indicator 802 of stream prefetch request 136 equalsa value of 0. If so flow proceeds to block 2232; otherwise, flowproceeds to block 2228.

At block 2228, memory subsystem 104 aborts the stream prefetch request136, because in the embodiment shown, only the values 0 through 5 arevalid values for the cache_level indicator 802. Flow ends at block 2228.

At block 2232, memory subsystem 104 generates a request to bus interfaceunit 102 to prefetch the cache line containing the location specified bycurrent_prefetch_address 324 in stream prefetch request 136 from thesystem memory into one of the caches of memory subsystem 104 based onthe urgency field in locality indicator 804 of FIG. 8 and upon thememory subsystem 104 configuration, i.e., on the number of caches, theirrelationship in the hierarchy, and the size of each cache. In anotherembodiment, stream prefetch engine 202 also bases the choice ofdestination cache of memory subsystem 104 on thestream_fetch-ahead_distance 612 of FIG. 6. In another embodiment, streamprefetch engine 202 also bases the choice of destination cache of memorysubsystem 104 on the stream_prefetch_hysteresis value 818 of FIG. 8.Flow ends at block 2232.

Referring now to FIG. 23, a flowchart illustrating in detail block 1202of FIG. 12 according to the present invention is shown. Flow begins atdecision block 2302.

At decision block 2302, memory subsystem 104 of FIG. 1 determineswhether the ephemerality field value of locality indicator 804 of FIG. 8of stream prefetch request 136 of FIG. 1 equals a value of 0. If so flowproceeds to block 2304; otherwise, flow proceeds to decision block 2306.

At block 2304, memory subsystem 104 generates a request to bus interfaceunit 102 of FIG. 1 to prefetch the cache line containing the locationspecified by current_prefetch_address 324 of FIG. 3 in stream prefetchrequest 136 from the system memory into a cache of memory subsystem 104with an early eviction policy. Flow ends at block 2304.

At decision block 2306, memory subsystem 104 determines whether theephemerality field value of locality indicator 804 of stream prefetchrequest 136 equals a value of 1. If so flow proceeds to block 2308;otherwise, flow proceeds to decision block 2312.

At block 2308, memory subsystem 104 generates a request to bus interfaceunit 102 to prefetch the cache line containing the location specified bycurrent_prefetch_address 324 in stream prefetch request 136 from thesystem memory into a cache of memory subsystem 104 with a normaleviction policy. Flow ends at block 2308.

At decision block 2312, memory subsystem 104 determines whether theephemerality field value of locality indicator 804 of stream prefetchrequest 136 equals a value of 2. If so flow proceeds to block 2314;otherwise, flow proceeds to block 2316.

At block 2314, memory subsystem 104 generates a request to bus interfaceunit 102 to prefetch the cache line containing the location specified bycurrent_prefetch_address 324 in stream prefetch request 136 from thesystem memory into a cache of memory subsystem 104 with a late evictionpolicy. Flow ends at block 2314.

At block 2316, memory subsystem 104 aborts the stream prefetch request136, because in the embodiment shown, only the values 0 through 2 arevalid values for the ephemerality field of locality indicator 804. Flowends at block 2316.

Although the present invention and its objects, features and advantageshave been described in detail, other embodiments are encompassed by theinvention. In addition to implementations of the invention usinghardware, the invention can be implemented in computer readable code(e.g., computer readable program code, data, etc.) embodied in acomputer usable (e.g., readable) medium. The computer code causes theenablement of the functions or fabrication or both of the inventiondisclosed herein. For example, this can be accomplished through the useof general programming languages (e.g., C, C++, JAVA, and the like);GDSII databases; hardware description languages (HDL) including VerilogHDL, VHDL, Altera HDL (AHDL), and so on; or other programming and/orcircuit (i.e., schematic) capture tools available in the art. Thecomputer code can be disposed in any known computer usable (e.g.,readable) medium including semiconductor memory, magnetic disk, opticaldisk (e.g., CD-ROM, DVD-ROM, and the like), and as a computer datasignal embodied in a computer usable (e.g., readable) transmissionmedium (e.g., carrier wave or any other medium including digital,optical or analog-based medium). As such, the computer code can betransmitted over communication networks, including Internets andintranets. It is understood that the invention can be embodied incomputer code (e.g., as part of an IP (intellectual property) core, suchas a microprocessor core, or as a system-level design, such as a Systemon Chip (SOC)) and transformed to hardware as part of the production ofintegrated circuits. Also, the invention may be embodied as acombination of hardware and computer code.

Also, although the memory subsystem has been described with respect toparticular configurations, one skilled in the art will appreciate thatthe applicability of the cache_level and locality parameters is notlimited to a particular configuration. Furthermore, although themicroprocessor has been described as synchronizing stream prefetcheswith load and store instructions, one skilled in the art will appreciatethat stream prefetches may be synchronized with other memory accessinstructions in microprocessors whose instruction sets do not includeexplicit load and store instructions. Additionally, although a streamdescriptor has been described that enables a programmer to specifyvariable-sized stream blocks separated by a stream block stride, otherdata streams may be specified, such as a data stream with a compoundstride. For example, the stream prefetch engine may fetch N blocks atstride K followed by M blocks at stride L and then repeat. Additionally,other more complex data streams may be described by the streamdescriptor, such as trees and graphs. Furthermore, the stream lengthcould be described in the stream descriptor as the number of streamblocks in the data stream. Alternatively, the stream length could bedescribed as the actual number of bytes in the data stream template,i.e., the number of bytes in the subset, rather than as the number ofcontiguous bytes between the first and last byte of the data stream.Similarly, the stream_fetch-ahead_distance could be described as thenumber of bytes in the data stream template, i.e., the number of bytesin the subset, to fetch ahead rather than as the number of contiguousbytes to fetch ahead. Additionally, the stream descriptor may specify aninstruction stream, i.e., a stream of instructions, rather than a datastream, for prefetching into the instruction cache of themicroprocessor. Finally, rather than synchronizing stream prefetchingimplicitly, by monitoring loads and stores for hits in the data stream,explicit synchronization could be accomplished by adding new load andstore instructions to the instruction set (or additional bits toexisting load and store instructions) that when executed explicitlytrigger the stream prefetch engine to advance, thereby eliminating theneed for a stream hit detector. The new instructions would include astream_ID parameter (returned by the stream prefetch instruction) forspecifying which of the stream prefetch engines to trigger.Alternatively, normal load and store instructions could be assumed toadvance a predetermined one of the stream prefetch engines, and newinstructions would explicitly advance the other stream prefetch engines.A variation of the explicit stream prefetch trigger instructions in amicroprocessor that includes a stream hit detector is to add a bit tothe normal load and store instructions which, if set, instructs thestream prefetch engine not to trigger prefetching, which might be usefulwhen the programmer knows he needs to access something in the datastream once, but wishes to avoid triggering a prefetch that would bringdata into the cache prematurely or unnecessarily, at the expense ofevicting more important data.

Finally, those skilled in the art should appreciate that they canreadily use the disclosed conception and specific embodiments as a basisfor designing or modifying other structures for carrying out the samepurposes of the present invention without departing from the spirit andscope of the invention as defined by the appended claims.

1. A microprocessor coupled to a system memory by a bus, themicroprocessor comprising: an instruction decode unit, for decoding aninstruction, said instruction specifying a data stream in the systemmemory and a stream prefetch priority; a load/store unit, coupled tosaid instruction decode unit, for generating load/store requests totransfer data between the system memory and the microprocessor; a streamprefetch unit, coupled to said instruction decode unit, for generating aplurality of prefetch requests to prefetch said data stream from thesystem memory into the microprocessor, wherein said prefetch requestsspecify said stream prefetch priority; and a bus interface unit (BIU),coupled to said stream prefetch unit and said load/store unit, forgenerating transaction requests on the bus to transfer data between thesystem memory and the microprocessor in response to said load/storerequests and said prefetch requests, wherein said BIU prioritizes saidbus transaction requests for said prefetch requests relative to said bustransaction requests for said load/store requests based on said streamprefetch priority.
 2. The microprocessor as recited in claim 1, whereinsaid BIU includes an arbiter, for prioritizing said bus transactionrequests for said prefetch requests relative to said bus transactionrequests for said load/store requests based on said stream prefetchpriority.
 3. The microprocessor as recited in claim 2, wherein said BIUincludes a request buffer, for storing said load/store requests and saidprefetch requests, wherein said arbiter prioritizes said requests storedin said request buffer based on said stream prefetch priority.
 4. Amethod for prefetching a data stream into a microprocessor from a systemmemory coupled to the microprocessor by a bus, the method comprising:decoding an instruction, said instruction specifying the data stream anda stream prefetch priority; generating a plurality of load requests toload data from the system memory into the microprocessor in response toexecution of a plurality of load instructions; generating a plurality ofprefetch requests to load portions of the data stream from the systemmemory into the microprocessor in response to said decoding; andprioritizing said prefetch requests relative to said load requests fortransmission on said bus based on said stream prefetch priority.
 5. Acomputer program product for use with a computing device, the computerprogram product comprising: a computer usable storage medium, havingcomputer readable program code embodied in said medium, for causing amicroprocessor coupled to a system memory by a bus, said computerreadable program code comprising: first program code for providing aninstruction decode unit, for decoding an instruction, said instructionspecifying a data stream in the system memory and a stream prefetchpriority; second program code for providing a load/store unit, coupledto said instruction decode unit, for generating load/store requests totransfer data between the system memory and the microprocessor; thirdprogram code for providing a stream prefetch unit, coupled to saidinstruction decode unit, for generating a plurality of prefetch requeststo prefetch said data stream from the system memory into themicroprocessor, wherein said prefetch requests specify said streamprefetch priority; and fourth program code for providing a bus interfaceunit (BIU), coupled to said stream prefetch unit and said load/storeunit, for generating transaction requests on the bus to transfer databetween the system memory and the microprocessor in response to saidload/store requests and said prefetch requests, wherein said BIUprioritizes said bus transaction requests for said prefetch requestsrelative to said bus transaction requests for said load/store requestsbased on said stream prefetch priority.